Integrating Language, Vision and Action for Human Robot Dialog Systems
نویسندگان
چکیده
Developing a robot system that can interact directly with a human instructor in a natural way requires not only highly-skilled sensorimotor coordination and action planning on the part of the robot, but also the ability to understand and communicate with a human being in many modalities. A typical application of such a system is interactive assembly for construction tasks. A human communicator sharing a common view of the work area with the robot system instructs the latter by speaking to it in the same way that he would communicate with a human partner.
منابع مشابه
Combining Planning and Dialog for Cooperative Assembly Construction
This article presents work on integrating a dialog component with a reactive planner into an assembly construction system. Using a re-active planner with a tight connection to the dialog component enables the system to ask for user assistance when encountering execution failures. We developed an action detection based on vision to monitor the execution of robotic and human actions. The combinat...
متن کاملEye Spy: Improving Vision through Dialog
Despite efforts to build robust vision systems, robots in new environments inevitably encounter new objects. Traditional supervised learning requires gathering and annotating sample images in the environment, usually in the form of bounding boxes or segmentations. This training interface takes some experience to do correctly and is quite tedious. We report work in progress on a robotic dialog s...
متن کاملRobot Motion Vision Part II: Implementation
The idea of Fixation introduced a direct method for general recovery of shape and motion from images without using either feature correspondence or optical flow [1,2]. There are some parameters which have important effects on the performance of fixation method. However, the theory of fixation does not say anything about the autonomous and correct choice of those parameters. This paper presents ...
متن کاملSymbol Grounding for Robot Dialog Research Proposal
Advancements in robotics have led to an ever-growing repertoire of software capabilities (e.g., recognition, mapping, and object manipulation). However, robotic capabilities grow, the complexity of operating and interacting with such robots increases (such as through speech, gesture, scripting, or programming). Language-based communication can offer users the ability to work with physically and...
متن کاملIn IJCAI - 99 Workshop " Planning and Scheduling meet Real - Time Monitoring in a Dynamic and Uncertain
This article presents work on integrating a dialog component with a reactive planner into an assembly construction system. Using a re-active planner with a tight connection to the dialog component enables the system to ask for user assistance when encountering execution failures. We developed an action detection based on vision to monitor the execution of robotic and human actions. The combinat...
متن کامل